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(54) Methods and apparatus for visually identifying an area on a photograph or image where 
digital data is stored 

(57) Methods and apparatus for generating images 
and/or photographs from digital data files with data, for 
example, audio data, stored on the photograph are 
described. Ink which is invisible or almost invisible to the 
human eye but which can be detected using an optical 
reader is used for printing the data on the photograph. 
When the ink used for printing audio data is slightly 
noticeable to a human, the color or colors of the photo 
are adjusted to mask the visual effect of the ink. In order 
to identify the presence and/or location of printed audio 
data, the audio data region (50) on the photograph is 
identified using a non-destructive technique. In one 
embodiment, the optical density of all or a portion of the 
image area corresponding to the audio data region of a 
photograph is reduced slightly from that of the original 
image to identify the presence and location of the 
printed audio data (40). This approach identifies the 
presence and location of audio data without destroying 
the content of the image on which the audio data is 
printed. In another embodiment, audio data markers 
(70), for example, in the form of small lines or marks, are 
printed on the photograph to identify the presence 
and/or location of printed audio data. To avoid destroy- 
ing the image content of the photograph the audio data 
region identification marks may be printed on the edges 
or boarder of the photograph. The audio data may be 
read and played using a hand held scanner. 




FIG. 4 
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Description 

FIELD OF THE INVENTION 

[0001] The present invention relates to image and/or i 
photograph generation and, more particularly, to meth- 
ods and apparatus for generating images and/or photo- 
graphs with data, for example, audio data, stored on the 
image and/or photograph. 

BACKGROUND OF THE INVENTION 

[0002] As the price of digital cameras continues to 
decrease, they are becoming ever more popular. Many 
digital still cameras have the capability of recording 
audio signals as well as a digital image. The recording 
of audio sounds, for example, music at a wedding, is 
often optional. When audio signals are recorded, they 
are normally saved within the camera in a digital audio 
file which is associated with one of the digital image files 
also stored in the camera. Alternatively, the audio and 
associated image data may be stored in a single file 
with the audio and/or video data being identified by a file 
header. 

[0003] Thus, for each optical image captured by a dig- 
ital camera, audio data may, optionally, be associated 
therewith. 

[0004] Digital audio and image files generated using a 
digital camera can, and often are, downloaded to a 
computer system capable of displaying the image files 
and playing the audio files. In addition, many computer 
systems include image processing and printing capabil- 
ities. 

[0005] Unfortunately, at the present time, known sys- 
tems do not provide a satisfactory way of storing 
recorded audio information, associated with a photo- 
graph, directly on the photograph. 
[0006] In view of the above, it is apparent that there is 
a need for methods and apparatus which will allow the 
storing of audio data directly on a printed photograph. It 
is desirable that any such methods do not significantly 
interfere with the quality of the printed image or block 
out portions of the image. In addition, it is desirable that 
it be relatively easy for a person to visually detect the 
presence or absence of audio data on a photograph. It 
is also desirable that a person be able to visually identify 
the region of a printed image or photograph where the 
audio data is stored, for example, to facilitate the read- 
ing back and playing of the audio data via, for example, 
the use of a hand held scanner. 

SUMMARY OF THE INVENTION 

[0007] The present invention resides in a method of 
generating a printed image, the method characterized 
by the steps of: processing data representing an image 
to add visible printed data identification indicia to the 
image; printing both the image including the visible 



printed data identification indicia and data to be stored 
on the printed image, the data to be stored being printed 
in a manner that results in non-visible machine readable 
data being included on the printed image. 
> [0008] The present invention also resides in a printed 
image, characterized by: a printing substrate; a first 
layer of ink representing non-visible machine readable 
data; a layer of visible ink representing an image, the 
layer of visible ink including visual indicia of the pres- 
10 ence of non-visible machine readable data. 

[0009] The present invention further resides in an 
apparatus for processing and printing an image and 
data, the image processing system characterized by: 
means for selecting an area within the image to print the 
75 data; means for processing the image to add visual indi- 
cia indicating the location of the selected area; and 
printing means for printing both the image including the 
visual indicia of the location in the image of the selected 
area and for printing the data in the selected area in a 
20 manner that renders it invisible to the human eye. 

[0010] In accordance with one aspect of the present 
invention, audio data is stored, for example, recorded, 
on a printed image or photograph by printing thereon a 
series of marks which can be optically detected and 
25 converted back into digital data representing, for exam- 
ple, audio information and/or other information, for 
example, creation date/time information, and so forth In 
one embodiment, a bar code type series of lines of var- 
ying thickness and/or spacing is used to represent the 
30 digital audio data. 

[0011] While the methods and apparatus of the 
present invention are suitable for use when generating a 
hard copy from any image or photograph data file, for 
purposes of brevity, the present invention will be 
35 explained in the context of printing a photograph. 

[001 2] So that the audio data printed on a photograph 
in accordance with the present invention does not 
degrade the visual image quality of the photograph or 
block out portions of the image, in one embodiment, the 
40 audio data is printed using ink which is not visually 
detectable by the human eye but which can be detected 
using an optical reader. 

[001 3] In another embodiment where the ink may be 
slightly noticeable to a human, for example, the printing 

45 of the ink used to represent audio data will have a slight 
effect on the perceived color of portions of photographs 
over which the audio data is printed, the color or colors 
of the photograph are adjusted to mask the effect of the 
ink used to print the audio data. 

so [0014] Normally the audio data printed on the image 
or photograph will cover only a small portion of the 
image or photograph. The area on a photograph in 
which the audio data is printed is referred to herein as 
the audio data region. 

55 [001 5] In order to facilitate the reading back of stored 
audio data, it is desirable that the audio data region of a 
photograph be visually identified. The identifying of the 
audio region facilitates subsequent placement of a 
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scanner when reading the stored data and avoids the 
need for a playback device to scan the entire image to 
read back the audio data and/or other recorded informa- 
tion. 

[0016] In one embodiment of the present invention, 
the audio data region is identified by subtly modifying 
the image area of the photograph where the audio data 
is stored. In such an embodiment, the photographic 
content in the audio data region is modified to render 
the region visibly identifiable without destroying the pho- 
tographic content printed in the audio data region. In 
one particular implementation, this is done by modifying 
the optical density of the photograph in the audio data 
region, for example, to lighten the region by a small 
amount such as 2-5%. 

[0017] In another embodiment, the audio data region 
of a photograph is marked by placing visual markers in 
a boarder region around the image being printed. In one 
embodiment this involves placing white boxes in a black 
boarder to identify the area within the image where the 
audio data is stored. In another embodiment this 
involves placing, for example, small black lines, on a 
white border placed around the photograph. 
[001 8] The visual indicia of stored audio data added to 
a photograph in accordance with the present invention 
make it easy to detect the presence and location of oth- 
erwise invisible printed data on a photograph or printed 
image. 

[0019] In addition to the above described embodi- 
ments and features, numerous additional features, 
embodiments and advantages of the methods and 
apparatus of the present invention are discussed below. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0020] 

Figure 1 illustrates a photographic system imple- 
mented in accordance with the present invention. 
Figure 2 is a flow chart illustrating the steps of a 
image generation routine implemented in accord- 
ance with the present invention. 
Figure 3 illustrates a photographic image which can 
be captured and represented as a digital data file. 
Figure 4 illustrates the photographic image shown 
in Fig. 3 with an audio data file printed thereon. 
Figure 5 illustrates a printed photograph generated 
in accordance with a first embodiment of the 
present invention wherein an audio data region on 
the photograph is identified by modifying the optical 
density of the audio data region. 
Figure 6 illustrates a printed photograph, generated 
in accordance with a second embodiment of the 
present invention, which includes a dark boarder 
with white areas used to indicate the location of 
stored data within the photograph. 
Figure 7 illustrates a printed photograph, generated 
in accordance with a third embodiment of the 



present invention, which includes a white boarder 
with black lines used to indicate the location of 
stored data within the photograph. 

5 DETAILED DESCRIPTION OF THE INVENTION 

[0021] As discussed above, the present invention is 
directed to methods and apparatus for generating 
images and/or photographs with data, for example, 

10 audio data, stored on the image and/or photograph. 
[0022] Figure 1 illustrates a system, generally indi- 
cated by the reference numeral 10, suitable for imple- 
menting the method of the present invention. The 
system 10 includes a digital camera 11, an image 

is processing system 20 and input/output devices in the 
form of a keyboard/mouse 30, a display device 32 and a 
printer 34. The display device 32 may be a color monitor 
while the printer 34 may be any one of a plurality of 
known digital printers capable of printing on a substrate, 

20 e.g, paper. While the printer 34 is illustrated as a single 
printer, it is to be understood that it may comprise one or 
more printers, for example, with each printer being 
responsible for printing a different color and one printer 
being responsible for printing the audio data using what 

25 is referred to as Invisible" ink The invisible ink is a color 
agent that, for the most part; does not absorb or fluo- 
resce light in the human visible spectrum but which is. 
nevertheless, visible to optical reading machines. In one 
embodiment, the ink used to print the audio data is 

30 invisible to the naked eye but can be excited by ultra vio- 
let light. Examples of invisible ink suitable for printing 
the audio data include ink that is based on organic com- 
plexes of, rare earth elements ions (lanthanides), such 
as: europium, gadolinium and terbium dipicolinates. 

35 The printer used to apply the invisible ink may be a con- 
ventional printer, for example, ink jet, dot matrix, impact, 
and so forth 

[0023] The digital camera 1 1 includes a flash unit 18, 
a viewing lens or digital image display area 12, a lens 16 

40 for receiving light from the image to be photographed, 
and a flash unit 18. In addition, the digital camera 11 
includes a microphone 14 used for recording audio data 
and first and second switches in the form of buttons 13, 
15. The first switch 13 is used to control the recording of 

45 audio data while the second switch 15 is to control the 
taking of a picture. Audio data which is recorded at the 
time a picture is taken is stored in an audio data file and 
associated with a digital image data file. The data in the 
digital image data file represents the picture taken dur- 

50 ing the audio recording time period. The size of each 
created audio data file is a function of the duration of the 
audio recording that is made by controlling the switch 
13. 

[0024] In the illustrated embodiment, the camera 1 1 is 
55 illustrated as being coupled, via a bus 17, to an 
input/output (I/O) interface of the image processing sys- 
tem 20. Via the bus 17, which may be coupled to the 
camera 11 via a interface terminal, digital audio and 
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image data files can be downloaded from the camera 1 1 
to the image processing system 20. 
[0025] The image processing system 20 of the 
present invention comprises the I/O interface 22, a cen- 
tral processing unit (CPU) 29 and a memory device 24. t 
The I/O interface 22 is responsible for interfacing the 
various components of the image processing system 20 
to the digital camera 1 1 and the various input/output 
devices 30, 32, 34 coupled to the image processing sys- 
tem 20. 

[0026] The memory 24 includes one or more image 
processing routines 26 implemented in accordance with 
the present invention. The image processing routines 
26 are used to control the CPU 29 when processing 
data files representing photographs and the audio files 
associated therewith. 

[0027] In addition to the image processing routines 26, 
the memory 24 includes an area 28 used for storing 
audio and image data received from the camera 1 1 , and 
for storing data files representing rendered images. A 
rendered image is an image data file which has been 
processed and formatted for printing. A rendered image 
may comprise a plurality of two dimensional data sets, 
each data set corresponding to a different color to be 
rendered. In the case where audio data is to be printed 
using an invisible ink the invisible ink is treated as an 
additional color to be rendered. Accordingly, in render- 
ing color images with which an audio data file is associ- 
ated, the CPU 29 will generate from the data received, 
for example, from the camera 1 1 , a separate data file 
corresponding to each of the colors cyan, magenta, and 
yellow (assuming use of a CMY color space) or red, 
green and blue (assuming use of an RGB color space), 
in addition to a data file corresponding to the invisible 
ink to be used to print the audio data on the photograph. 
[0028] Referring now to Fig. 2, the steps of an image 
generation routine 40 implemented in accordance with 
an exemplary embodiment of the present invention are 
illustrated. The steps illustrated in Fig. 2 may be per- 
formed by various components of the system illustrated 
in Fig. 1 including the digital camera 1 1 and CPU 29. 
[0029] The routine 40 begins in step 41 wherein the 
routine is initiated. Operation progresses to steps 42 
and 44 wherein audio data and optical data is captured, 
respectively. Steps 42 and 44 may be performed in par- 
allel by the digital camera 11 illustrated in Fig. 1 
Fig. 3 illustrates an image which can be captured and 
represented as an optical data file, for example, by per- 
forming step 44. 

[0030] From steps 42 and 44, operation proceeds to 
step 45 wherein the optical data captured by the camera 
1 1 and the captured aucfio data associated with the opti- 
cal data, if any, are transferred to the memory 24 of the 
image processing system 20. 

[0031] After the transfer of data to be processed to the 
memory 24, in step 46 a determination is made by the 
image processing system to determine if audio data 
was received in addition to image processing data. This 



determination may be made by, for example, examining 
a header included in a received data file to determine if 
it includes audio data in addition to optical data or, alter- 
natively, determining if separate optical and audio data 
5 files were received from the digital camera 1 1 . 

[0032] In step 46, rf it is determined that optical data 
was received without associated audio data, operation 
proceeds to step 48 wherein the received optical image 
data is rendered in memory, as is known in the art, in 
w the desired color space, for example, CMY or RGB. As 
part of the rendering process, the optical data is proc- 
essed to form a separate set of two dimensional data for 
each of the colors to be printed, in the case where 
image rendering is performed in step 48, no audio data 
75 needs to be printed on the photograph being generated. 
Accordingly, in step 48 three distinct sets of color print- 
ing data, each representing a two dimensional array of 
color printing information, will be generated. 
[0033] From step 48, operation proceeds to step 50 
20 wherein the rendered data, in the form of data files cor- 
responding to each color to be printed, are sent from the 
portion 28 of memory where they are stored prior to 
being supplied to the printer, for example, via bus 19. 
[0034] If in step 46 it was determined that audio data 
25 was received in addition to optical data, for example, the 
header of a received data file indicates the presence of 
both audio and optical data in the file or an audio data 
file was received in addition to an optical data file, oper- 
ation proceeds from step 46 to step 52. 
30 [0035] In step 52 a determination is made, for exam- 
ple, by the CPU 29, as to the size of the image area to 
be used to store the audio data on the photograph in 
printed form. Accordingly in step 52 the size of the audio 
data region is determined. Normally the size of the 
35 audio data region will correspond to, or be approxi- 
mately the size of, the minimum area which can be used 
to store the contents of the received audio data file. The 
size of the audio data area may be determined, for 
example, by dividing the amount of the received audio 
40 data, for example, in kilobytes (KB) by the data density, 
for example, (KB/sq. inch) at which the audio data is to 
be printed on the photograph. However, it can be made 
larger than the minimum size required. For example, if 
pre-selected unit sizes, for example, !/£ square inch 
45 blocks, representing fixed units of data are used when 
determining the size of the audio data region, the audio 
data region would be selected to match the smallest 
multiple of the fixed size units that could accommodate 
the recording of the audio data. 
so [0036] Once the size of the audio data region is deter- 
mined, the image, represented by the received optical 
data, is analyzed. In one embodiment, the analysis is 
performed to detect an image area of the determined 
size, in the image or photograph to be printed, which 
55 has the highest optical density, for example, average 
optical density. In this manner, a relatively dark region of 
the image, matching the size of the audio data region, is 
identified for use in printing of the audio data. 
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[0037] As discussed above, the ink used to print the 
audio data is invisible, or mostly invisible, to the human 
eye. Unfortunately, depending on the ink used for print- 
ing the audio data, it may not be completely invisible. In 
fact, the printing of the dye to store the audio irtforma- 5 
tion may slightly impact the perceived color of the 
underlying image over which the ink is printed. By 
selecting a dark or relatively dark region of the image 
being printed to be used as the audio data region, the 
visible impact of the ink used to print the audio data can 
be minimized. 

[0038] Once the area to be used for recording the opti- 
cal data has been determined, for example, selected in 
step 54 by performing the optical data analysis, opera- 
tion proceeds to step 56. In step 56 optical data coordi- 
nates, that is, two dimensional data coordinates 
corresponding to an area of the image to be printed, are 
assigned for the recording of the audio data on the pho- 
tograph being printed. In addition, the spatial require- 
ments for rendering the audio data are specified, for 
example, a two dimensional point in the photograph 
being printed is assigned for the position of one of the 
corners of the audio data region and the overall size and 
shape of the audio data region on the photograph is 
specified. 

[0039] Referring now to Fig. 4, there is illustrated the 
image of Fig. 3 with audio data printed thereon in the 
form of a bar code 40 located towards the upper right 
portion of the photograph. While the bar code 40 is 
printed using visible ink in Fig. 4, in accordance with the 
present invention it would normally be printed using 
invisible ink. The area, in which the bar code 40 is 
located is the region which is specified in step 56. 
[0040] From step 56 operation proceeds to step 58 
wherein the optical data received in step 45 is proc- 
essed to render the data suitable for outputting, for 
example, to a printer. In this step the received optical 
image data is rendered in memory in the desired color 
space, for example, CMY or RGB. As part of the render- 
ing process, the optical data for each color to be output 
is processed to form a separate set of two dimensional 
data. Accordingly, in step 58 three distinct sets of color 
data, each representing a two dimensional array of 
information, for example, printing information, is gener- 
ated when preparing color image data to be output. In 
step 58, the rendering process may include adjusting 
the intensity of the individual colors being rendered in 
the audio data region to compensate for any effect that 
printing of the audio data over the color image may have 
on the colors as they are perceived by a person viewing 
the photograph. Since the ink used for printing the audio 
data will normally have been selected prior to perform- 
ing step 58, its perceived effect, if any. on the colors 
being printed will normally be known from prior empiri- 
cal tests and/or general printing knowledge. This makes 
the modification of the colors in the audio data region to 
conceal the effect of the printed audio data possible. 
[0041] In step 60, the sets of optical image data ren- 



dered in memory during step 58 are further processed 
to suitably identify the presence and/or location of audio 
data in the photograph being printed. 
[0042] In a first embodiment, the presence and loca- 
tion of audio data is identified by reducing the optical 
density of the image in the audio data region. This 
involves, for example, decreasing the optical density of 
each of the 3 rendered color image sets by a subtle but 
visually identifiable amount, for example, 2-3% in the 
audio data region defined in step 56. In this manner, the 
photograph generated will have an easy to identify 
audio data recording while the integrity of the photo- 
graph is maintained since all the image data is dis- 
played and visible. 

[0043] Fig. 5 illustrates the image of Fig. 3, after it is 
printed in accordance with the method of the present 
invention which involves modifying the optical density of 
the audio data region 50 to identify the presence and 
location of printed audio data. Note that in Fig. 5, the 
printed audio data is invisible. However, the audio data 
region 50 located in the upper right portion of Fig. 5 is 
readily identifiable as the light area of the image, that is, 
the area within the flag. 

[0044] As an alternative to adjusting the optical den- 
sity of the photograph being printed in the audio data 
region, in accordance with one embodiment of the 
present invention in step 60, the optical data region is 
identified by adding a neutral density, for example, 
black, boarder 62 to the photograph with white marks, 
for example, blocks 60 being located in the boarder 62 
to identify the location within the photograph of the 
audio data region. The result of this method of identify- 
ing the audio data is shown in Fig. 6. In this particular 
embodiment, the need to modify the image the content 
of the image (optical density) is avoided via the use of a 
boarder with identifying marks. 

[0045] In yet another embodiment, the result of which 
is illustrated in Fig. 7, a white boarder with relatively 
small marks 70 is added to the photograph to identify 
the presence and placement of printed audio data in the 
photograph. In such an embodiment, the marks 70 are 
generated by processing one or more of the data sets 
representing the rendered color image to add the 
boarder and audio region identification marks 70. 
[0046] From step 60, operation proceeds to step 62 
wherein the audio data is rendered in memory for print- 
ing using the optical data coordinates and spatial infor- 
mation specified in step 56. For rendering and printing 
purposes, the audio data is treated as simply another 
two dimensional image, for example, bar code, to be 
printed using a different ink. Normally the audio data is 
printed on top of the colors used to represent the image 
portion of the photograph being printed. In accordance 
with the present invention, the rendering of the audio 
data results in the creation of a set of data in memory 28 
that is used to control the printer 34 to print the audio 
data at the image location corresponding to the audio 
data region. 
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[0047] After the completion of step 62, the rendered 
optical and audio data sets are output, in step 50, for 
example, to the printer 34. In step 64, the printer 34 
prints the image and/or the image is displayed in 
response to the received data sets. In a CMY printing 
embodiment, first the cyan is printed using the data set 
representing the cyan portion of the image being 
printed. The cyan is followed by the printing of the 
magenta and yellow portions of the image respectively. 
Finally, the rendered set of audio data is used to print 
the audio data. As discussed above, the audio data is 
printed using invisible or nearly invisible ink selected for 
this purpose. 

[0043] In the case of an RGB embodiment, the colors 
are output in the order red, green, blue, with the audio 
data being output last. 

[0049] After the image is printed or displayed in 
accordance with the present invention, the routine of the 
present invention stops in step 66. pending the re-start- 
ing of the routing 40 and the capture of a new image. 
[0050] While the methods and apparatus of the 
present invention have been described in the context of 
processing audio and image data captured using a dig- 
ital camera, the image processing and printing methods 
of the present invention can be used for processing 
audio and image data files regardless of their source. 
Furthermore, the methods and apparatus of the present 
invention can be used for locating, identifying and print- 
ing different types of information and not merely audio 
information on a photograph in a manner that is invisible 
or largely invistole to the naked eye. 
[0051] In adcStion, while the Fig. 1 embodiments 
shows the image processing system 20 as a separate 
device located external to the camera 11, in another 
embodiment, the image processing system 20 is incor- 
porated directly into the camera 1 1 . 

Claims 



optical density of the processed image portion. 

3. The method of claim 2, wherein the processed 
image portion corresponds to at least a portion of 

5 an image area in which the data to be stored is to 

be printed. 

4. The method of claim 3, wherein the processing of 
the portion of the image is characterized by the step 

10 of reducing the optical density of the processed 
image portion by approximately 2-3%. 

5. The method of claim 1 , wherein the step of process- 
ing data representing an image to add visible 

is printed data identification indicia to the image is 
characterized by the step of: 

processing a portion of the image correspond- 
ing to at least a portion of the area in which the 
20 data to be stored is to be recorded to brighten 

the portion of the image being processed. 

6. The method of claim 1 , wherein the step of process- 
ing data representing an image to add visible 

25 printed data identification indicia to the image is 
characterized by the step of : 

adding a boarder to the image; and 
including in the boarder visible marks (70) iden- 
30 tifying the location within the printed image of 

at least a portion of the non-visible machine 
readable data. 

7. The method of claim 6, wherein the boarder is a first 
35 color and the visible marks are a second color. 

8. The method of claim 7, wherein the first color is 
black and the second color is white. 



A method of generating a printed image, 
method characterized by the steps of: 



the 40 9. The method of claim 7, wherein the first color is 
white and the second color is black. 



2. 



processing data representing an image to add 
vistole printed data identification indicia to the 
image; 

printing both the image including the visible 
printed data identification indicia (40) and data 
to be stored on the printed image, the data to 
be stored being printed in a manner that results 
in non-visible machine readable data being 
included on the printed image. 

The method of claim 1 , wherein the step of process- 
ing data representing an image to add visible 
printed data identification indicia to the image is 
characterized by the step of: 

processing a portion of the image to reduce the 



45 



50 



55 



1 0. The method of claim 1 , wherein the step of process- 
ing data representing an image to add visible 
printed data identification indicia to the image is 
characterized by the step of: 

adding visible marks identifying the location 
within the printed image of at least a portion of 
the non-visible machine readable data. 

11. The method of claim 10, wherein the visible marks 
are added to an outer peripheral portion of the 
image. 

1 2. The method of claim 1 , further characterized by the 
step of: 
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determining the size of the area to be used for 
recording the data to be stored as a function of 
the amount of data to be stored. 

13. The method of claim 12, further characterized by 
the step of: 

identifying an area of the image of the deter- 
mined size having a high optical density rela- 
tive to other areas of the image. 

14. The method of daim 13, wherein the step of print- 
ing the data to be stored is characterized by the 
step of: 

printing the data to be stored in the identified 
area. 

15. The method of claim 13, wherein the step of 
processing data representing the image, further 
characterized by the step of: 

modifying one or more image colors in the 
identified area of the image to compensate for 
any effect the printing of the data to be stored 
will have on the perceived color of the printed 
image. 

1 6. The method of claim 1 , wherein the step of process- 
ing data representing the image, further character- 
ized by the step of: 

modifying one or more image colors in the 
identified area of the image to compensate for 
any effect the printing of the data to be stored 
will have on the perceived color of the printed 
image. 

17. The method of claim 16, 

wherein the data is audio data, and 
wherein the step of processing data represent- 
ing an image is further characterized by the 
steps of: 



data 

19. The printed image of claim 18, wherein the visual 
indicia of the presence of non-visible machine read- 

5 able data indicates the location on the printing sub- 

strate of at least a portion of the non-visible 
machine readable data. 

20. The printed image of claim 19, wherein the visual 
10 indicia of the presence of non- visible machine read- 
able data is a lightened area within the image. 

21. The printed image of claim 19, wherein the visual 
indicia of the presence of non-visible machine read- 

15 able data are marks in a boarder forming the out- 
side of the image. 

22. The printed image of claim 19, wherein the non-vis- 
ible machine readable data is audio data. 

20 

23. The printed image of claim 22, wherein the printed 
image is a photograph. 

24. An apparatus for processing and printing an image 
25 and data, the image processing system character- 
ized by: 

means for selecting an area within the image to 
print the data; 

30 means for processing the image to add visual 

indicia indicating the location of the selected 
area; and 

printing means for printing both the image 
including the visual indicia of the location in the 
35 image of the selected area and for printing the 

data in the selected area in a manner that 
renders it invisible to the human eye. 

25. The apparatus of claim 24, further characterized by: 

40 

a digital camera coupled to the selecting and 
processing means, the digital camera generat- 
ing the image to be printed from an optical 
input and generating the data to be printed 
45 from an audio input 



20 

23. 



30 



35 



selecting, as a function of the optical den- 
sity of the image, a location within the 
image where the data to be stored is to be 
printed. 

50 

18. A printed image, characterized by: 
a printing substrate; 

a first layer of ink representing non-visible 
machine readable data ; 55 
a layer of visible ink representing an image, the 
layer of visible ink including visual indicia of the 
presence of non-visible machine readable 



7 



3NSDOCID: <EP„ 091 3989A2_I_> 



EP 0 913 989 A2 



12 



14 




.18 



10 



11 



17- 



19 



16 



20 



24 



IMAGE PROCESSING 
SYSTEM 

22 
Z_ 



I/O 

INTERFACE 



MEMORY 



26 



n 

28 



IMAGE 
PROCESSING 
ROUTINES 



DATASTORAGE/ 
IMAGE RENDERING 
MEMORY SPACE 



CPU 



29 



KEYBOARD / 
/MOUSE W 



30 



DISPLAY 
DEVICE 



32 

V 



PRINTER 



34 



FIG. I 



8 



3NSDOCID: <EP_ 



.091 3989 A2J_> 



EP 0 913 989 A2 

J 



42 

JL_ 



START \ 

— 



41 



CAPTURE AUDIO 
DATA 



TRANSFER VIDEO AND/ 
OR AUDIO DATA TO 
IMAGE PROCESSING 
SYSTEM 



48 



45 



RENDER IMAGE 
IN MEMORY IN 
DESIRED COLOR 
SPACE 



50 



46« 



DETERMINE 
IF RECEIVED DATA 
INCLUDES AUDIO DATA 
IN ADDITION TO 
OPTICAL DATA. 



60- 



PROCESS RENDERED IMAGE 
DATA TO GENERATE VISUAL 
INDICIA OF PRESENCE AND 
LOCATION OF AUDIO DATA 
REGION 



62 



OUTPUT 
RENDERED 
DATA 



PRINT/ 
DISPLAY 
IMAGE 

[ 



RENDER AUDIO DATA IN 
MEMORY USING ASSIGNED 
OPTICAL DATA 
COORDINATES AND 
SPECIFIED SPACIAL 
REQUIREMENTS 



STOP^ ) 



66 



64 



FIG. 2 



CAPTURE 
OPTICAL DATA 
REPRESENTING 

AN IMAGE 



DETERMINE SIZE OF 
IMAGE AREA 

REQUIRED TO PRINT 
AUDIO DATA 



ANALYZE OPTICAL IMAGE 

DATA TO DETECT 
MAXIMUM DENSITY AREA 
OF IMAGE 
CORRESPONDING TO 
DETERMINED SIZE 



ASSIGN OPTICAL DATA 
COORDINATES AND 

SPECIFY SPATIAL 
REQUIREMENTS FOR 
RENDERING AUDIO DATA 



56 



7 



58 



PROCESS OPTICAL DATA 
TO RENDER IN MEMORY 

OPTICAL DATA IN 
SPECIFIED COLOR SPACE 



9 



3NSDOCID: <EP 0913989A2_I_> 



EP 0 913 989 A2 





FIG. 3 



BNSDOCID: <EP 0913989A2J_> 



10 



EP 0 913 989 A2 





FIG. 4 



3NSDOCID: <EP 091 3989 A2_l_> 



11 



EP 0 913 989 A2 




3NSDOCID: <EP 0913989A2J_> 



12 




8NSDOCID: <EP 0913989 A2_l_> 



EP 0 913 989 A2 




FIG. 7 



3NSDOCID: <EP 091 3989 A2_l_> 



14 



(19) 



J 



(12) 



(88) Date of publication A3: 

17.05.2000 Bulletin 2000/20 

(43) Date of publication A2: 

06.05.1999 Bulletin 1999/18 

(21 ) Application number: 98203451 .4 

(22) Date of filing: 16.10.1998 



Europaisches Patentarrrt 
European Patent Office 
Off ice europSen des brevets (11) EP0913 989 A3 

EUROPEAN PATENT APPLICATION 

(51) Int. CI. 7 : H04N 1/32 



(84) Designated Contracting States: 


(72) Inventor: 


AT BE CH CY DE DK ES Fl FR GB GR IE IT LI LU 


Soscia, Peter P., 


MCNLPT SE 


c/o Eastman Kodak Company 


Designated Extension States: 


Rochester, New York 14650-2201 (US) 


AL LT LV MK RO SI 






(74) Representative: 


(30) Priority: 28.10.1997 US 959041 


Lewandowsky, Klaus, Dipl.-lng. et al 


(71) Applicant: EASTMAN KODAK COMPANY 


Kodak Aktiengesellschaft, 


Patentabteilung 


Rochester, New York 14650 (US) 


70323 Stuttgart (DE) 



CO 

< 

CO 

o 

CO 

o> 
o 
a. 

LLI 



(54) Methods and apparatus for visually identifying an area on a photograph or image where 
digital data is stored 

(57) Methods and apparatus for generating images 
and/or photographs from digital data files with data, for 
example, audio data, stored on the photograph are 
described. Ink which is invisible or almost invisible to the 
human eye but which can be detected using an optical 
reader is used for printing the data on the photograph. 
When the ink used for printing audio data is slightly 
noticeable to a human, the color or colors of the photo 
are adjusted to mask the visual effect of the ink. In order 
to identify the presence and/or location of printed audio 
data, the audio data region (50) on the photograph is 
identified using a nondestructive technique. In one 
embodiment, the optical density of all or a portion of the 
image area corresponding to the audio data region of a 
photograph is reduced slightly from that of the original 
image to identify the presence and location of the 
printed audio data (40). This approach identifies the 
presence and location of audio data without destroying 
the content ol the image on which the audio data is 
printed. In another embodiment, audio data markers 
(70), for example, in the form of small lines or marks, are 
printed on the photograph to identify the presence 
and/or location of printed audio data. To avoid destroy- 
ing the image content of the photograph the audio data 
region identification marks may be printed on the edges 
or boarder of the photograph. The audio data may be 
read and played using a hand held scanner. 




FIG. 4 



Printed by Xerox (UK) Business Services 
2.16.7 (HRSJ/3.6 



BNSDOCID: <EP 091 3989 A3_l_> 



EP 0 913 989 A3 



European Patent 
Office 



EUROPEAN SEARCH REPORT 



Applicatton Number 

EP 98 20 3451 



DOCUMENTS CONSIDERED TO BE RELEVANT 




Category 


Citation of document with indication, where appropriate, 
of relevant passages 


Relevant 
to claim 


CLASSIFICATION OF THE 
APPLICATION (lnLCt.6) 


X 
Y 


US 4 889 367 A (MILLER L MICHAEL) 
26 December 1989 (1989-12-26) 
* the whole document * 


18 

1,6,10, 
19,21 


H04N1/32 


Y 


EP 0 3G0 729 A (EMPIRE AIRPORT SERVICE KK) 
25 January 1989 (1989-01-25) 
* the whole document * 


1,6,10, 
19,21 




A 


US 4 627 819 A (BURROWS ROGER 1) 
9 December 1986 (1986-12-09) 
* the whole document * 


1-5,17, 
22 




A 


EP 0 670 555 A (OLYMPUS OPTICAL CO) 
6 September 1995 (1995-09-06) 
* abstract; figures 2A.2B * 


24,25 




A 


US 5 291 243 A (HECKMAN DEAN A ET AL) 
1 March 1994 (1994-03-01) 

* abstract * 

US 4 430 563 A (HARRINGTON JOHN A) 
7 February 1984 (1984-02-07) 

* the whole document * 


3,4 






6-11 


TECHNICAL FIELDS 
SEARCHED (lnl.CI.6) 


A 


HG4N 

G06K 


A 

A 


1 July 1992 (1992-07-01) 

* abstract; claims 1-3; figures 1-4 * 


3-5 




A 


EP 0 411 602 A (0MR0N TATEISI ELECTRONICS 
CO) 6 February 1991 (1991-02-06) 
* claims 1-9 * 


6 




A 


EP 0 490 457 A (ENSCHEDE & Z0NEN GRAFISCH) 
17 June 1992 (1992-06-17) 
* abstract * 






The present search report has been drawn up for all claims 







THE HAGUE 



Dale of completion of the search 

30 March 2000 



Hanratty, C 



CATEGORY OF CITED DOCUMENTS 

X : particularty relevant if taken alone 

Y : particularly relevant if combined with another 

document of the same category 
A : technological background 
O : non-written disclosure 
P : intermediate document 



T : theory or principle underlying the invention 
E : earlier patent document, but published on, or 

after the filing date 
D : document cited in the application 
L : document cited for other reasons 

& : member of the same patent family, corresponding 
document 



2 



WSDOCID: <EP 091 3989 A3_l_> 



1 



EP 0 913 989 A3 



ANNEX TO THE EUROPEAN SEARCH REPORT 

ON EUROPEAN PATENT APPLICATION NO. EP 98 20 3451 



This annex lists the patent family members relating to the patent documents cited in the above-mentioned European search report. 
The members are as contained in the European Patent Office EDP file on 

The European Patent Office is in no way liable for these particulars which are merely given for the purpose of information. 

30-03-2000 



Patent document 




Publication 


Patent family 




Publication 


cited in search report 




date 




m©mber(s) 




date 


US 4889367 


A 


26-12-1989 


NONE 








EP 0300729 


A 


25-01-1989 


JP 


1023383 


A 


26-01-1989 




* 
A 


no i o 1 AQ£ 


NONE 








EP 067Q555 


A 


06-09-1995 


AU 


668987 


B 


23-05-1996 








AU 


4833793 


A 


26-04-1994 








US 


5896403 


A 


20-04-1999 








JP 


11316808 


A 


16-11-1999 








JP 


11328303 


A 


30-11-1999 








WO 


9408314 


A 


14-04-1994 








JP 


11007511 


A 


12-01-1999 








JP 


11007512 


A 


12-01-1999 








JP 


10187907 


A 


21-07-1998 








JP 


10261059 


A 

A 


29-09-1998 








JP 


10187908 


A 


21-07-1998 








JP 


1 01 87909 


A 


21-07-1998 








JP 


10187910 

X \J X KJ 9 -/ X \J 


A 


21-07-1998 








JP 


10187873 

X \J X KJ / W / ij 


A 


21-07-1998 








JP 


2833975 


B 


09-12-1998 








JP 


6231466 


A 


19-08-1994 


US 5291243 


A 


01-03-1994 


DE 


69420753 


D 


28-10-1999 








DE 


69420753 


T 


09-03-2000 








EP 


0610035 


A 


10-08-1994 








JP 


6258982 


A 


16-09-1994 


US 4430563 


A 


07-02-1984 


NONE 








EP 0493091 


A 


01-07-1992 


JP 


4335480 


A 


24-11-1992 


EP 0411602 


A 


06-02-1991 


JP 


3063168 


A 


19-03-1991 


EP 0490457 


A 


17-06-1992 


NL 


9002740 


A 


01-07-1992 








AT 


136848 


T 


15-05-1996 








CA 


2057479 


A 


14-06-1992 








DE 


69118837 


D 


23-05-1996 








DE 


69118837 


T 


12-12-1996 








DK 


490457 


T 


26-08-1996 








ES 


2089118 


T 


01-10-1996 








GR 


3020558 


T 


31-10-1996 








HK 


1006957 


A 


26-03-1999 








IL 


100352 


A 


27-02-1994 








JP 


6152928 


A 


31-05-1994 








US 


5374976 


A 


20-12-1994 



Si For more details about this annex ; see Official Journal of the European Patent Office, No. 12782 



BNSDOCID: <EP 09 13989 A3 J_> 



